Near Real-time Data Warehousing with Multi-stage Trickle & Flip

A data warehouse typically is a collection of historical data designed for decision support, so it is updated from the sources periodically, mostly on a daily basis. Today’s business however asks for fresher data. Real-time warehousing is one of the trends to accomplish this, but there are a number of challenges to move towards true real-time. This paper proposes ‘Multi-stage Trickle & flip’ methodology for data warehouse refreshment. It is based on the ‘Trickle & flip’ principle and extended in order to further insulate loading and querying activities, thus enabling both of them to be more efficient.